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Charles  K.  Parsons 
Donald  B.  Fedor 

School  of  Management 
Georgia  Institute  of  Technology 

This  research  project  had  as  its  goals  the  development,  validation,  and  field  testing 
of  new  measures  of  individual  differences  which  assess  people's  propensities  to  seek, 
generate,  or  interpret  performance  feedback  information  in  a  particular  way.  Specifically, 
based  on  preliminary  work,  it  was  thought  that  internal  and  external  propensities  exist 
which  make  individuals  more  or  less  likely  to  prefer,  rely  on,  seek,  or  attend  to  primarily 
internally  or  externally  generated  performance  cues.  These  propensities,  if  identified  and 
measured,  should  be  related  to  skill  acquisition,  performance  improvement,  self-regulatory 
processes,  performance  maintenance,  as  well  as  a  variety  of  affective  and  cognitive 
responses  to  performance  settings  based  on  the  interaction  of  the  performer’s  feedback 
predispositions  and  the  characteristics  of  the  feedback  available. 

If  individual  differences  in  feedback  propensities  influence  the  shaping,  processing 
and  responding  to  one’s  feedback  environment,  then  there  are  several  important 
applications  to  which  we  can  apply  such  understanding.  For  example,  some  tasks  or  jobs 
are  inherently  richer  in  the  feedback  environment  which  they  provide,  and  thus  may  be 
better  matched  to  the  proclivities  of  some  performers  than  others.  Some  training  methods 
rely  more  on  self-learning  than  others,  yet  individuals  may  differ  in  their  ability  to  benefit 
from  the  nature  of  the  feedback  provided  by  self-paced  instruction,  computer-based 
instruction,  or  simulator  training.  Some  individuals  may  need  supplemental  feedback  in 
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order  to  acquire  new  skills  or  improve  their  performance  in  a  given  situation,  while  others 
may  perform  well  with  the  available  feedback.  Some  individuals  may  be  better  able  to  self- 
generate  feedback  on  an  ongoing  basis  and  thus  be  better  able  to  operate  in  environments 
which  rely  on  self-assessments  and  self-correction  than  their  counterparts  who  seek  or 
desire  external  feedback  which  may  not  be  available.  Some  situations,  such  as  training 
simulators,  may  be  redesigned  so  as  to  provide  different  feedback  (e.g.,  process  only, 
outcome  only,  or  both)  to  different  performers  based  on  what  they  might  best  utilize. 
Finally,  it  may  be  possible  to  train,  induce,  or  otherwise  enhance  self-monitoring  behaviors 
which  lead  to  heightened  abilities  to  self-assess  and  self-regulate  performance. 

In  summary,  we  wanted  to  better  understand  the  role  of  predispositions  in 
explaining  how  different  individuals  go  about  shaping  their  feedback  environment, 
processing  feedback  information,  and  responding  to  such  information.  The  driving  belief 
behind  this  line  of  research  has  been  that  individuals  differ  in  ways  that  are  specific  to  their 
orientation  toward  performance  feedback  situations,  and  that  such  differences,  if  identified 
and  appropriately  measured,  would  be  valuable  in  better  understanding  the  links  between 
feedback  and  performance  as  well  as  other  outcomes  of  interest  (e.g.  feedback-seeking, 
satisfaction,  etc.). 

Background  and  phases  of  research  program 

The  most  obvious  way  in  which  people  may  differ  in  how  they  interact  with  their 
environment  for  performance  feedback  purposes  would  seem  to  be  along  some  internal- 
external  orientation  dimension.  Certainly  our  personal  experiences  suggest  that  some 
people  seem  “to  be  able  to  figure  things  out  for  themselves,”  while  others  seem  “clueless.” 
Some  people  seem  to  need  considerably  more  external  confirmation  of  their  performance 
than  others.  Some  people  welcome  and  even  seek  others’  assessments  of  their 
performance,  while  others  disdain  or  avoid  such  assessments.  Some  people  seem 
devastated  by  others’  negative  assessments  of  their  performance,  while  others  shrug  it  off 
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or  welcome  it  because  they  perceive  it  as  valuable.  Some  people  critically  evaluate  the 
outcomes  of  their  efforts,  while  others  seem  to  pass  it  off  without  much  thought  Some 
people  are  “hard  on  themselves”  when  it  comes  to  self-assessments  of  performance,  while 
others  seem  naively  optimistic  about  what  they  have  achieved.  Some  people  seem 
introspective  and  can  tell  exactly  where  and  why  their  performance  may  have  deviated  from 
a  standard,  while  others  need  to  have  it  pointed  out  to  them.  Some  trainees  seem  to  hang 
on  every  work  uttered  by  an  instructor  while  others  seem  to  tune  out  instruction  and  go 
their  own  way.  Some  students  can’t  wait  to  get  their  grades,  others  seem  blasd  or 
disinterested. 

Not  only  is  this  internal  vs.  external  orientation  distinction  based  on  everyday 
observations,  but  it  is  also  consistent  with  the  fact  that  when  individual  differences  have 
been  found  to  make  a  difference  in  feedback  research,  many  of  them  seem  to  relate  to 
relative  confidence  in  internal  vs.  external  capabilities,  causalities,  or  merit  (e.g.,  self¬ 
esteem,  self-efficacy,  locus  of  control).  Finally,  the  external-internal  distinction  has  been 
previously  used  in  the  feedback  literature  to  distinguish  between  feedback  sources  in  work 
settings  (Greller  &  Herold,  1975;  Hanser  &  Muchinsky,  1978;  Herold  &  Greller,  1977; 
Herold  &  Parsons,  1985).  Based  on  the  theoretical,  empirical,  and  practical  appeal  of  this 
distinction  as  the  most  obvious  one  to  pursue,  our  earlier  research  focused  on  whether  one 
or  more  individual  difference  dimensions  could  be  identified  which  would  characterize 
people’s  seeking  of,  reliance  on,  or  preference  for  either  internally  or  externally  mediated 
feedback.  Based  on  encouraging  results  from  our  early  work  (e.g.,  Herold  &  Parsons, 
1985;  Herold,  Parsons,  &  Rensvold,  1991;  Parsons  &  Herold,  1986),  we  proposed  a 
research  program  to  pursue  the  development,  refinement,  construct  validation,  and  field 
testing  of  measures  which  capture  these  individual  differences. 

The  research  program  can  be  divided  roughly  into  three  parts  or  phases,  with  some 
overlap  between  activities  across  phases  (each  phase  corresponded  roughly  to  Year  1,  Year 
2,  and  Year  3  tasks  in  the  original  proposal).  In  Phase  I,  we  focused  on  literature  review. 
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instrument  development,  and  construct  validation  of  our  early  measures  of  feedback 
propensities.  In  Phase  n  we  focused  on  laboratory  and  field  studies  to  demonstrate  the 
utility  of  the  new  measures  in  explaining  outcomes  of  interest  Also  in  Phase  II,  we 
engaged  in  a  major  effort  to  revise  and  update  our  measures  based  on  theoretical  and 
psychometric  considerations.  In  Phase  HI  we  focused  on  a  major  field  demonstration  of 
the  utility  of  our  measures  in  explaining  behavior  in  an  applied  setting. 

Since  Phase  I  and  II  work  has  been  extensively  described  in  earlier  interim  reports, 
this  report  will  only  summarize  that  work.  The  bulk  of  the  report  will  focus  on  the  final 
development  work  on  the  measures  and  on  the  field  demonstration  of  their  utility. 

PHASE  I 

This  phase  focused  on  literature  reviews  of  related  areas  (e.g.,  self-efficacy,  self¬ 
esteem,  performance  feedback,  feedback  seeking,  etc.)  and  on  the  initial  phases  of  the 
development  and  construct  validity  of  measures  intended  to  reflect  internal  or  external 
predispositions  toward  performance  feedback.  Specifically,  three  independent  dimensions 
or  predispositions  were  identified  empirically  and  confirmed,  using  confirmatory  factor 
analysis,  on  two,  widely  different  independent  samples.  The  three  dimensions  were; 
External  Propensity  reflecting  the  preference  for  externally  mediated  feedback  as  well  as 
greater  faith  in  such  information  than  that  which  one  can  self-generate;  Internal  Propensity 
which  reflects  the  opposite  preference  but  also  suggests  the  tendency  to  reconcile 
differences  between  internal  and  external  feedback  in  the  direction  of  the  internally 
generated  information;  and.  Internal  Ability  which  seems  to  distinguish  between  the  mere 
preference  for  internal  feedback  and  the  perceived  ability  to  accurately  generate  such 
feedback. 

Various  activities  aimed  at  supporting  the  construct  validity  and  demonstrating  the 
potential  utility  of  the  new  measures  were  also  conducted.  Using  a  sample  of  military 
helicopter  pilot  trainees,  we  found  theoretically  consistent  relationship  between  the  three 
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scales  and  need  for  achievement,  self-esteem  and  tolerance  for  ambiguity,  while  showing 
their  relative  independence  from  another  frequently  used  internal-external  construct,  locus 
of  control.  Using  the  same  sample,  we  demonstrated  that  External  Propensity  was  related 
to  both  the  seeking  of  feedback  from  instructors  and  “monitoring”  one’s  environment  for 
feedback  clues,  while  Internal  Propensity  was  negatively  related  to  instructors  assessments 
of  students’  performance  during  early  phases  of  the  training.  This  was  interpreted  as 
suggesting  that  internal  reliance  on  one’s  own  feedback  early  during  training  may,  in  fact, 
be  detrimental  to  learning. 

Several  computerized-task  laboratory  studies  were  also  conducted  during  this 
phase,  examining  the  seeking  of  process  and/or  outcome  information  on  the  part  of 
different  individuals.  Results  showed  that  for  novel  tasks,  both  Internal  and  External 
Propensity  were  positively  related  to  the  seeking  of  outcome  feedback,  while  Internal 
Propensity  was  negatively  related  to  the  seeking  of  process  feedback. 

PHASE  n 

This  phase  of  the  research  focused  on  a  series  of  small  studies  aimed  at 
investigating  the  utility  of  our  dimensions  for  explaining  behaviors  and  attitudes  under 
various  task  feedback  conditions.  During  this  time  we  also  initiated  a  major  effort  at 
revisiting  the  conceptual  clarity  of  our  scales  by  developing  a  theoretically-based  typology 
of  possible  dimensions  which  our  three  dimensions  might  be  tapping,  writing  new  items  to 
reflect  those  dimensions  not  adequately  covered  by  the  existing  scales,  and  analyzing  the 
total  set  of  items  (new  and  old)  to  investigate  the  stability  of  our  dimensions,  better 
ascertaining  their  meaning,  and  identifying  any  new  scales  or  subscales  which  reflect 
aspects  of  the  domain  of  all  possible  items  which  were  not  tapped  by  the  original 
dimensions. 


5 


Empirical  findings 


Several  studies  aimed  at  better  understanding  the  utility  of  our  measures  were 
reported  in  our  earlier  interim  reports.  In  one  study  we  investigated  the  seeking  of  process 
and  outcome  feedback  in  a  computer-based  experimental  task.  Results  showed  that  our 
Internal  Propensity  measure  was  associated  with  very  little  seeking  behavior  under 
conditions  of  high  task  familiarity,  but  with  high  levels  of  seeking  under  conditions  of  low 
task  familiarity.  This  finding  suggests  that  “internals”  will  seek  information  when  they  lack 
the  internal  standards  to  self-assess,  but  will  stop  doing  so  when  they  feel  that  they  are 
familiar  with  the  task.  Furthermore,  we  found  an  effect  for  social  presence,  such  that 
higher  levels  of  Internal  Propensity  were  associated  with  the  lowest  levels  of  feedback 
seeking_/ro/n  the  computer  when  an  experimenter  was  in  the  room,  whereas  those  low  in 
Internal  Propensity  sought  the  highest  levels  of  feedback  in  the  same  condition.  These 
results  suggest  that  social  cues  may  inhibit  feedback  seeking  on  the  part  of  those  who  are 
more  internally  reliant  for  feedback. 

In  another  study,  using  an  Army  marksmanship  simulation,  we  found  that  under 
conditions  of  blocked  simulator  feedback,  those  high  in  Internal  Propensity  did 
significantly  better  than  their  counterparts  early  in  the  training,  but  seemed  to  lose  interest  in 
the  task  when  denied  the  opportunity  to  confirm  their  performance  via  some  feedback. 
Under  the  usual,  computer-mediated  feedback,  those  “high”  in  Internal  Ability  performed 
significantly  better  than  their  “low”  counterparts.  Finally,  we  created  a  constrained  social 
feedback  condition  in  which  an  experimenter  conveyed  only  the  information  which  the 
computer  would  have  conveyed,  and  found  that  “high”  External  Propensity  subjects,  while 
starting  out  the  same  as  their  “low”  counterparts,  performed  progressively  worse  as  the 
experiment  continued.  We  interpreted  this  result  as  indicating  frustration  at  being  in  a 
setting  where  a  feedback  agent  was  available  but  would  not  interact  with  the  subject  other 
than  in  a  very  impersonal,  low  information  way. 
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In  general,  these  studies,  as  well  as  several  other  field  and  lab  studies  spanning 
several  years,  provided  strong  encouragement.  We  found  the  individual  differences  in 
feedback  styles  or  propensities,  as  measured  by  our  scales,  can  be  meaningfully  assessed, 
and  that  these  assessments  make  a  difference  in  various  learning  and  performance 
situations. 

Scale  refinements 

Based  on  the  series  of  studies  we  had  done,  we  were  encouraged  by  the  prospects 
for  being  able  to  reliably  assess  these  individual  differences  and  demonstrate  their  utility  in 
explaining  a  variety  of  person-situation  interactions  in  feedback-relevant  settings.  Before 
proceeding  further,  we  embarked  on  a  re-examination  of  the  feedback  propensities 
constructs,  trying  to  balance  our  earlier  inductive  approach  with  a  more  deductive  effort 
aimed  at  making  sure  that  the  domain  of  possible  feedback  items  was  adequately  sampled 
and  that  the  three  dimensions  did  not  reflect  omissions  attributable  to  the  original  pool  of 
items. 

In  order  to  do  that,  we  took  each  of  the  three  dimensions  (External  Propensity, 
Internal  Propensity,  and  Internal  Ability)  and  further  dimensionalized  them  according  to 
two  well-established  dimensions  along  which  feedback  is  thought  to  vary,  whether  it  is 
positive  or  negative,  and  whether  it  refers  to  outcomes  or  to  process,  and  crossed  the  four 
resulting  categories  with  whether  the  item  reflected  a  feedback-relevant  cognition,  an 
attitude,  or  a  behavior.  This  resulted  in  twelve  categories  of  feedback  items  into  which  the 
original  items  from  each  scale  were  divided,  and  a  new  item-writing  effort  was  initiated  to 
identify  items  which  would  reflect  each  cell  which  was  not  represented  or  under¬ 
represented. 

The  resulting  items  were  administered  to  404  undergraduate  and  graduate  students 
for  the  purpose  of  identifying  underlying  factors.  We  found  that  the  original  External 
Propensity  dimension,  which  consisted  of  items  expressing  a  preference  for  extemally- 
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mediated  feedback  when  such  was  expected  to  be  largely  neutral  or  positive  in  nature, 
remained  intact,  but  that  parallel  items  written  to  reflect  the  valuing  of  negative  external 
feedback  formed  their  own  factor,  including  items  which  reflected  the  purposeful  seeking 
of  external  feedback  even  if  one  expects  it  to  be  negative.  This  yielded  two  external 
propensity  scales:  External  Propensity  —  neutral/positive,  and  External  Propensity  — 
negative. 

The  original  Internal  Feedback  Propensity  dimension  it  wiU  be  recalled  reflected  a 
tendency  for  self-feedback  to  dominate  information  from  external  sources.  This  dimension 
again  remained  intact,  but  a  new  set  of  internal  items  formed  a  dimension  which  reflects 
feedback  of  a  self-congratulatory  type  when  performance  warrants  it;  we  termed  this 
dimension  Self-Reinforcement  Propensity. 

Finally,  the  original  Internal  Feedback  Ability  dimension  reflecting  one’s  belief  that 
he/she  can  figure  things  out  for  themselves,  or  accurately  generate  their  own  feedback  also 
remained  intact,  but  new  items  formed  an  additional  dimension  which  seems  to  reflect 
one’s  confidence  in  one’s  ability  to  self-assess  without  the  aid  of  feedback  from  others  - 
we  called  this  dimension  Internal  Feedback  Confidence.  All  scales  had  adequate  internal 
consistencies  (alphas  ranging  from  .68  to  .84  with  only  Internal  Feedback  Confidence 
being  below  .70).  An  examination  of  their  relationship  with  self-esteem,  tolerance  for 
ambiguity,  need  for  achievement  and  locus  of  control  showed  them  to  be  theoretically 
consistent,  but  not  redundant  with  other  commonly  used  measures  of  individual  differences 
(Parsons,  Herold,  Fedor,  Rensvold,  Goodman  &  Davis,  1994). 
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PHASE  III 

For  this  phase  we  set  out  to  conduct  a  large-scale  field  study  in  which  the  utility  of 
our  measures  could  be  studied  under  “real  world”  conditions  in  which  feedback  was  being 
provided  as  part  of  a  regular  routine  designed  for  training  purposes.  Since  this  would  be 
the  final  phase  of  a  prolonged  exploratory  research  program,  it  was  desirable  to  study 
people  in  controlled  feedback  conditions.  While  one  may  lose  something  in  generalizability 
of  findings  to  settings  where  feedback  may  vary  by  source,  by  type  (e.g.,  outcome  vs. 
feedback),  or  by  other  features  (e.g.,  timing,  sign,  specificity,  etc.),  it  was  felt  that  getting 
a  sounder  understanding  of  person-situation  interactions  in  a  limited,  yet  not  contrived 
setting  was  worth  it  Toward  that  end,  we  sought  a  field  site  in  which  we  could  clearly 
identify  the  performance  feedback  available  and  study  people’s  utilization  of,  and  reactions 
to  that  feedback.  It  was  also  thought  that  periods  of  skill  acquisition  might  be  most 
appropriate  for  studying  people’s  reliance  on,  or  utilization  of  various  feedback  available  in 
their  environment.  The  identification  of  a  private  flight  academy,  and  their  utilization  of  a 
standardized,  computer-based  flight  simulation  as  an  integral  part  of  their  training,  met 
these  criteria. 

The  study  took  place  at  the  COMAIR  aviation  training  facility  where  student  pilots 
receive  flight  training  and  flight  certifications.  Students  at  this  facility  participate  in 
orientation,  simulator  training,  ground  school,  and  in-flight  cockpit  training.  Upon 
entering  the  training  program,  students  participate  in  an  orientation  session  which  oudines 
basic  training  procedures.  Students  then  spend  approximately  one  to  two  weeks  training 
on  a  Basic  Flight  Instruction  Tutorial  System  (BFITS).  This  simulator  provides:  (1)  basic 
instruction  on  principles  of  flight  (e.g.  aerodynamics)  and  (2)  practice  in  a  simulated 
cockpit  setting.  Following  successful  completion  of  BFITS  training,  students  either  enter 
ground  school  or  move  into  in-flight  cockpit  training  with  an  instructor  pilot 

The  BFITS  simulator  is  a  PC-based  flight  training  system  consisting  of  31  lessons 
requiring  about  50  hours  to  complete  on  average  (Benton,  Corriveau,  Koonce,  &  Tirre, 
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1992).  BFITS  combines  a  tutorial  that  teaches  the  declarative  knowledge  (facts, 
definitions,  concepts)  with  a  procedural  trainer  that  teaches  the  procedural  knowledge 
(cognitive  and  motor  skills)  needed  to  perform  basic  flight  maneuvers  in  a  simulated 
general  aviation  aircraft  Following  the  basic  flight  principles  segment  (declarative 
knowledge),  the  first  phase  of  flight  simulation  in  BFITS  training  teaches  basic  procedures, 
including  climb,  straight  and  level  flight,  descent,  and  turns.  Subsequent  phases  go  into 
more  complex  maneuvers  which  make  use  of  the  basic  procedures.  Trials  to  mastery  for 
each  of  the  maneuvers  are  recorded  as  measures  of  performance. 

During  the  simulation  no  outside  feedback  of  any  type  is  provided.  Instructors 
have  not  yet  been  assigned  to  the  students,  and  other  than  questions  concerning  the 
operation  of  the  hardware  and  software,  which  can  be  directed  to  the  lab  administrator,  no 
sources  of  information  are  available  to  the  student  while  performing.  The  computer 
provides  no  process  feedback,  but  only  lets  students  know  when  they’ve  exceeded  any 
lesson’s  parameters,  causing  a  failure  and  the  need  to  repeat  that  lesson.  The  computer- 
provided  feedback  can  probably  be  characterized  as  negative  for  the  vast  majority  of 
students.  This  is  because  most  BFITS  lessons  are  complex  enough  such  that  multiple 
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errors  are  committed  and  multiple  trials  are  required  to  pass  a  lesson.  While  formal 
feedback  is  not  available  beyond  that  which  the  computer  provides,  students  probably  do 
discuss  their  simulator  experience  with  others,  students  and/or  academy  staff,  outside  the 
formal  training  setting.  This  feedback,  however,  needed  to  be  sought  out  by  the  students 
and  had  to  be  informal  in  nature  since  the  feedback  providers  were  not  privy  to  the  actual 
performance.  To  the  degree  that  such  feedback  seeking  went  on,  it  provided  an 
opportunity  for  us  to  study  whether  of  such  behavior  related  to  our  feedback  propensities. 

With  the  cooperation  of  COMAIR,  we  had  access  to  15  entering  classes  of 
approximately  20  students  each,  over  a  period  of  15  months.  On  the  first  day  of  orientation 
we  would  hold  a  group  administration  of  our  questionnaires  containing  the  individual 
differences  measures,  various  other  measures  of  general  attitudes,  expectations  concerning 
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the  training,  and  demographics.  BFITS  training  would  commence  immediately  after 
orientation.  Subsequent  to  the  completion  of  BFITS,  the  lab  coordinator  would  place  a 
post-BFITS  questionnaire  in  students’  mailboxes,  encouraging  them  to  return  these  to 
him/her  in  sealed  envelopes  for  transmission  to  the  research  team.  The  post-BFITS 
questionnaire  contained  measures  of  attitudinal  and  behavioral  reactions  to  the  simulator 
experience.  Students  were  informed  that  their  participation  was  totally  voluntary  and  that 
their  individual  data  would  be  treated  in  strictest  confidence  and  would  not  be  shared  with 
the  academy,  or  anyone  else.  Student  identification  numbers  were  provided  by  the  students 
to  enable  us  to  link  various  data  collection  efforts  over  time. 

After  students  completed  their  flight  instruction  phase,  culminating  in  the  granting 
of  a  private  pilot’s  license,  we  had  access  to  their  flight  records  for  the  purpose  of  obtaining 
data  concerning  the  number  of  hours  it  took  for  them  to  successfully  complete  the  certificate 
training.  Not  all  students  entering  the  academy  were  in  the  private  pilot  phase.  Those  in 
more  advanced  phases,  such  as  instruments,  multi-engine,  etc.  were  followed  for  the  pre- 
and  post-BFITS  data  in  order  to  study  any  differences  attributable  to  flight  experience.  For 
the  private  pilot  students  we  obtained  data  from  their  flight  records  on  the  hours  it  took 
them  to  complete  the  course.  Since  the  advanced  students  split  into  many  different 
programs,  of  varied  length,  and  the  cohorts  were  relatively  small,  we  did  not  track  them 
beyond  their  reactions  to  the  simulator  (which  was  required  of  all  students  regardless  of 
their  flight  experience).  This  procedure,  coupled  with  normal  and  expected  attrition  at  the 
various  data  collection  points  (e.g.,  failure  to  return  questionnaires),  plus  the  elimination  of 
students  from  foreign  countries  who  were  having  obvious  language  difficulties,  created  a 
sample  of  181  for  whom  we  have  pre-  and  post-BFITS  data,  and  86  students  for  whom  we 
have  both  sets  of  BFITS  data,  as  well  as  flight  performance  data. 


Measures 


An  initial  survey  was  administered  to  participants  during  the  orientation  session. 

This  survey  assessed  demographic  information  and  individual  differences  in  feedback- 
related  propensities.  The  full  version  of  these  scales  appear  in  Appendix  1.  All  items  used 
a  5  point  agiee-disagree  response  scale. 

INTERNAL  FEEDBACK  ABILITY  -  A  10  item  scale  (internal  consistency  alpha  = 
.85)  measured  participants’  perceptions  of  their  ability  to  accurately  generate  their 
own  feedback.  Sample  items  from  this  scale  include:  "When  I  finish  something,  I 
can  usually  tell  right  away  whether  I  did  it  well  or  not  and  If  I  make  a  mistake 
while  working,  I  can  usually  sense  it  immediately". 

INTERNAL  FEEDBACK  CONFIDENCE  -  A  5  item  scale  (alpha  =  .64)  measured 
participants’  confidence  in  their  ability  to  assess  their  own  performance  without  the 
aid  of  feedback  from  others.  Sample  items  from  this  scale  include:  "When  others' 
opinions  about  my  work  are  different  than  my  own,  I  tend  to  question  my  own 
judgment"  and  "I  find  that  I  am  not  very  good  at  assessing  my  own  performance 
and  need  to  rely  on  the  inputs  of  others"  (both  reversed  scored). 

INTERNAL  FEEDBACK  PROPENSITY  -  A  5  item  scale  (alpha  =  .73)  measured 
the  extent  to  which  participants  value  their  own  assessments  more  than  the  feedback 
of  others,  and  reconciled  contradictory  feedback  in  favor  of  one  s  self.  Sample 
items  from  this  scale  include:  "How  other  people  view  my  work  is  not  as  important 
as  how  I  view  my  own  work"  and  "As  long  as  I  think  I  have  done  something  well, 
I  am  not  too  concerned  about  how  other  people  think  I  have  done". 
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PROPENSITY  TO  SELF-REINFORCE  -  A  5  item  scale  (alpha  =  .76)  measured 
the  extent  to  which  participants  tended  to  reflect  on  positive  performance 
experiences.  Sample  items  from  this  scale  include:  "I  tend  to  give  myself  a  pat  on 
the  back  for  a  job  well  done"  and  "I  like  to  step  back  and  reflect  on  a  job  well 
done". 

EXTERNAL  FEEDBACK  PROPENSITY  (POSITIVE/NEUTRAL)  -A  6  item  scale 
(alpha  =  .71)  measured  the  extent  to  which  participants  tended  to  value  feedback 
from  other  people.  Sample  items  include:  "It  is  very  important  for  me  to  know 
what  people  think  of  my  work"  and  "I  like  being  told  how  well  I  am  doing  on  a 
project". 

EXTERNAL  PROPENSITY  FOR  NEGATIVE  FEEDBACK-  A  4  item  scale  (alpha 
=  .77)  measured  the  extent  to  which  participants  tended  to  seek  out  and  value 
negative  feedback  from  other  people.  Sample  items  include:  "I  seek  out  reactions  to 
my  work  even  if  I  think  they  might  be  negative"  and  "I  seek  others’  assistance  in 
figuring  out  how  to  improve  my  performance". 

A  second  survey  was  administered  after  each  participant  completed  BFITS.  On  this 
survey  we  assessed  a  set  of  participant  reactions  to  BFITS.  These  scales  also  appear  in 
Appendix  1. 

FEEDBACK  SEEKING  -  A  7  item  scale  (alpha=  .74)  assessed  the  extent  to  which 
participants  sought  information  from  other  people  during  their  BFITS  training 
experience.  Sample  items  include:  "I  asked  for  help  from  other  BFITS  trainees" 
and  "I  talked  to  others  (including  individuals  not  doing  BFITS)  about  the 
usefulness  of  BFITS”. 
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FEEDBACK  DESIRE  -  A  3  item  scale  (alpha  =  .74)  assessed  the  extent  to  which 
the  participant  would  have  liked  more  feedback  from  external  sources  during 
BFITS  training.  Sample  items  include:  "I  wished  I  had  someone  working  with  me 
to  provide  additional  help"  and  "I  would  have  liked  for  someone  to  periodically 
review  my  progress". 

INSTRUMENTALITY  -  A  4  item  scale  (alpha  =  .93)  that  assessed  the  belief  that 
BFITS  would  be  beneficial  in  flight  training.  Sample  items  include:  "BFTTS  helped 
me  identify  potential  weaknesses  in  my  flight  skills"  and  "BFTTS  will  make  me  a 
better  pilot". 

STRESS  -  A  3  item  scale  (alpha  =  .60)  assessed  the  degree  to  which  the  participant 
felt  frustrated  and  stressed  during  BFITS  training.  Sample  items  include  "I  found 
BFITS  flight  lessons  to  be  frustrating"  and  "I  felt  stress  while  going  through  flight 
lessons". 

Along  with  the  demographic  variables,  individual  difference  measures,  and  BFTTS 
reactions  collected  through  surveys,  performance  measures  were  collected  during  both 
simulator  and  in-flight  training. 

SIMULATOR  PERFORMANCE  -  The  number  of  trials  subjects  took  to 
successfully  perform  all  of  the  BFITS  flight  maneuvers  lessons  was  used  as  the 
measure  of  simulator  performance. 

IN-FLIGHT  TRAINING  PERFORMANCE  -  In-flight  performance  was  defined 
as  the  number  of  in-flight  hours  it  took  to  complete  certification  for  the  Private  Pilot 
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license.  In-flight  training  time  was  recorded  in  the  participants’  training  record  by 
the  students'  instructor  pilot 

Our  results  will  be  presented  as  follows.  Following  a  description  of  the  sample  and 
overall  relationships  among  the  variables,  we  will  present  our  primary  results  from  in  two 
sections.  In  the  first  section,  we  will  focus  on  simulator  performance,  reactions,  and 
attitudes  based  on  the  post-simulator  data.  The  next  section  will  focus  on  the  private  pilot 
students  and  relationships  between  their  pre-  and  post-simulator  data  and  actual 
performance  in  the  cockpit. 

Table  1  shows  the  means  and  standard  deviations  from  the  demographic  variables 
and  other  variables  of  interest  in  this  study.  The  average  age  of  the  participants  was  26.3 
years  and  93%  were  male.  In  terms  of  previous  flight  experience,  about  24%  had  no  flight 
experience,  6%  had  some,  but  less  that  15  hours,  and  52%  had  a  private  license.  This 
latter  group  was  excluded  from  analysis  concerning  the  impact  of  our  individual  differences 
on  time  to  private.  Additionally,  flight  experience  was  a  control  variable  in  the  other 
analyses.  The  average  number  of  trials  required  to  complete  BFITS  was  350. 


Insert  Table  1  about  here 


Table  2  shows  the  correlations  among  the  individual  difference  variables,  BFITS 
performance,  time  to  private  pilot  license  (TMPRTV),  previous  flight  experience 
(FLTEXP),  and  various  reactions  to  the  BFITS  simulator.  The  correlation  between  BFTTS 
performance  and  time  to  private  is  .51.  Those  students  who  performed  well  in  simulator 
training  tended  to  perform  well  during  in-flight  training. 


Insert  Table  2  about  here 
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Table  2  also  shows  the  correlation  between  the  individual  differences  and  BFITS 
performance  and  time  to  private  license.  For  BFITS  performance,  we  see  that  there  is  a 
statistically  significant  rregative  correlation  with  Internal  Feedback  Confidence,  indicating 
that  people  with  such  confidence  take  fewer  trials  to  complete  BFITS.  Interestingly,  there 
is  also  a  significant  positive  correlation  between  Internal  Feedback  Confidence  and 
previous  flight  experience.  It  is  unclear  why  such  an  association  should  exist  unless  the 
internal  feedback  confidence  scale  was  completed  by  new  trainees  with  the  aviation  task  in 
mind.  In  this  case  it  would  make  sense  that  people  with  flight  experience  would  feel  more 
confident  in  the  feedback  they  might  generate  during  performance  of  aviation-related  skills. 

There  were  no  significant  correlations  between  the  individual  difference  variables 
and  time  to  private  license.  The  impact  of  the  feedback  propensity  variables  may  be  in  their 
interaction  with  performance  feedback.  This  will  be  explored  later. 

In  Table  2,  there  are  also  some  statistically  significant  correlations  between  various 
reactions  to  BFITS  and  the  individual  difference  variables,  especially  External  Feedback 
Propensity.  These  relationships  are  better  analyzed  using  multiple  regression  which  will 
follow  shortly. 

Finally,  we  are  also  interested  in  the  intercorrelations  of  the  individual  difference 
variables  themselves.  Because  these  scales  are  relatively  new,  we  are  still  looking  at 
evidence  concerning  their  properties  such  as  intercorrelations.  In  the  current  sample,  the 
correlations  range  from  -.12  to  .46,  supporting  the  idea  that  they  are  somewhat  independent 
dimensions.  Additionally,  their  internal  consistencies  continue  to  be  satisfactory  (ranging 
from  .64  to  .85,  See  Appendix  1). 

Simulator  results 

In  order  to  understand  the  unique  effects  of  flight  experience,  BFITS  performance 
and  individual  differences  on  various  outcome  measures,  we  used  hierarchical  multiple 
regression.  For  each  regression,  in  step  1  flight  experience  was  entered.  In  step  2,  BFITS 
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performance  was  entered.  In  step  3,  each  of  the  six  feedback-related  individual  difference 
measures  was  entered.  Step  4  examined  possible  interactions  between  BFl'l'S  performance 
and  individual  differences.  In  the  final  step,  we  conserved  degrees  of  freedom  by  using  a 
forward  entry  procedure  for  the  six  interaction  terms.  In  forward  entry,  each  variable  in  the 
block  of  variables  is  considered  one  at  a  time  for  entry  into  the  equation.  At  each  step,  the 
variable  with  the  smallest  probability  of  F  is  entered.  This  process  continues  until  none  of 
the  remaining  variables  makes  a  significant  contribution  to  the  regression  equation.  The 
above  procedure  was  followed  except  for  the  multiple  regression  for  BFITS  performance 
where  step  2  was  eliminated  because  BFITS  was  the  dependent  variable. 


BFITS  Performance.  As  shown  in  Table  3,  performance  on  BFITS  was  predicted 
by  the  students’  level  of  flight  experience,  Internal  Feedback  Confidence  (INFBCON)i  and 
Internal  Feedback  Propensity  (INFBPR).  Flight  experience  accounted  for  17  percent  of 
the  variance  and  the  propensities,  in  total,  accounted  for  an  additional  5  percent  For  this 
outcome,  there  were  no  interactions  tested. 


Insert  Table  3  about  here 


The  results  indicate  that  flight  experience  was  negatively  related  to  the  number  of 
trials  it  took  the  students  to  complete  BFITS.  Stated  another  way,  the  more  prior  flight 
experience  the  students  had  the  better  they  did  in  the  BFITS  simulator.  This  finding 
provides  some  validation  for  the  BFITS  simulator.  However,  flight  experience  accovmted 
for  less  variance  in  BFITS  performance  than  might  be  expected.  This  may  be  due  to  a 
number  of  factors.  First,  flight  experience  beyond  a  certain  level  might  not  constitute  any 
additional  advantage  or  that  one  might  need  to  reach  a  certain  level  of  experience  before  one 
gains  any  advantage  on  the  simulator.  An  inspection  of  the  mean  performance  levels  for 
the  five  different  levels  used  to  categorize  flight  experience  suggests  that  the  latter 
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explanation  appears  to  receive  some  support  Those  with  over  15  hours  of  flight  training 
seem  to  perform  much  better  than  those  with  less  flight  experience. 

Second,  given  that  BFITS  is  a  PC-based  training  system,  the  skills  necessary  for 
good  performance  are  not  going  to  perfectly  match  those  that  contribute  to  successful 
cockpit  performance.  Third,  former  flight  training  might,  at  least  partially,  interfere  with 
learning  from  BFITS.  The  advance  students  already  "know"  how  to  fly  and  revisiting  the 
basic  flight  skills  on  BFITS  may  have  required  some  relearning  or  unlearning  to  perform 
well.  Finally,  we  heard  from  the  students  that  there  were  some  frustrating  lessons  in 
BFITS.  This  might  have  been  particularly  so  for  the  students  with  significant  flight 
experience.  Therefore,  the  experienced  students  may  have  had  to  deal  with  stronger 
emotions  than  the  less  experienced  students.  Some  indirect  support  for  this  last  notion  can 
be  drawn  from  the  fact  that  experience  level  was  negatively  related  to  seeing  BFITS  as 
instrumental  in  learning  actual  flight-related  skills.  This  finding  is  discussed  in  greater 
detail  below. 

Two  of  the  individual  differences  also  made  significant  contributions  to  predicting 
BFITS  performance  even  after  experience  level  was  taken  into  account.  Those  who 
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reported  higher  confidence  in  their  ability  to  self  generate  performance  feedback  {Internal 
Feedback  Confidence)  did  better  than  those  who  reported  low  confidence  in  this  ability. 
Possible  explanations  for  this  finding  are  that  those  high  on  this  individual  difference  were: 
1)  less  frustrated  by  the  simulator  feedback  which  tended  to  be  negative  because  of 
confidence  in  their  own  feedback,  2)  better  able  to  develop  their  own  cues  and  strategies  for 
effectively  using  the  available  feedback,  or  3)  more  effective  in  generating  accurate  process 
feedback  for  themselves  and  thus  able  to  self  correct  more  readily.  Overall,  it  does  appear 
that  those  high  on  Internal  Feedback  Confidence  were  better  able  to  make  the  external  cues 
more  useful. 

In  contrast,  those  high  on  Internal  Feedback  Propensity  (INFBPR)  encountered 
more  difficulties  in  BFITS  than  those  low  on  this  individual  difference.  This  finding  is 
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consistent  with  previous  results  in  another  flight  training  situation  reported  earlier  to  the 
Army  Research  Institute.  It  appears  that  the  higher  individuals  score  on  INFBPR  the  less 
likely  they  are  to  utilize  feedback  other  than  their  own.  Therefore,  they  either  overly  filter 
corrective  feedback  or  they  might  feel  they  are  doing  fine  even  when  other,  more  veridical 
sources  indicate  they  are  not 

Dpsifp.  For  Feedback.  Table  4  shows  the  multiple  regression  of  flight  experience, 
BFITS  performance,  and  the  individual  differences  on  Feedback  Desire.  Flight  experience 
clearly  makes  a  difference  in  how  much  additional  feedback  and  help  individuals  want 
The  variance  accounted  for  is  14  %  and  the  regression  weight  is  negative  indicating  that 
those  with  more  flight  experience  were  less  interested  in  additional  feedback.  In  contrast 
poorer  performance  in  BFITS  was  associated  with  a  greater  desire  for  feedback.  This 
variable  accounted  for  an  additional  5.7%  of  the  variance  in  feedback  desire.  Among  the 
individual  difference  variables,  the  block  of  6  variables  accounted  for  a  statistically 
significant  7.2%  of  additional  variance.  External  Feedback  Propensity  had  a  significant 
regression  weight  with  higher  scores  associated  with  greater  desire  for  feedback.  There 
were  no  significant  interactions  between  BFITS  performance  and  the  individual 
differences. 


Insert  Table  4  about  here 


Therefore,  we  have  found  evidence  that  in  the  feedback  environment  that  existed  for 
the  BFITS  simulator,  there  is  a  statistical  reliable  effect  for  the  individual  difference  that  is 
most  logically  associated  with  desiring  more  external  feedback  (External  Feedback 
Propensity).  This  is  further  evidence  that  there  is  a  disposition  towards  wanting  external 
feedback  and  it  plays  a  role  in  who  wants  such  feedback  several  weeks  into  a  training 
program.  This  is  especially  important  when  considering  that  many  training  simulators  are 
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designed  to  be  feedback  rich  environments,  but  only  from  a  single  source,  the  simulator 
itself.  An  identifiable  subset  of  trainees  desires  more  and  the  training  organization  may  be 
able  to  design  the  feedback  environment  to  better  accommodate  these  needs. 

Feedback  Seeking.  Self  reported  feedback  seeking  during  BFITS  training  was 
significantly  related  to  prior  flight  experience  and  two  of  the  individual  differences  (see 
Table  5).  In  addition,  the  desire  for  feedback  was  included  as  a  final  step  in  the  analysis. 
Flight  experience  accounted  for  just  over  7  percent  of  the  variance  and  the  propensities,  in 
total,  accounted  for  an  additional  8  percent.  For  this  outcome,  there  were  no  significant 
interactions.  The  desire  for  feedback  was  marginally  significant  (.07)  and  contributed 
approximately  2  percent  additional  variance. 


Insert  Table  5  about  here 


The  more  flight  experience  the  students  had  the  less  they  reported  engaging  in 

feedback  seeking.  This  finding  is  not  suiprising  given  that  they  probably  had  less  need  for 
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feedback  and  they  might  have  felt  more  foolish  asking  for  it  This  is  consistent  with 
Ashford's  work  that  has  pointed  to  the  resource  value  of  feedback  as  well  as  the  costs 
involved  in  obtaining  it  (Ashford,  1986). 

The  fact  that  External  Feedback  Propensity  (EXFBPR)  was  positively  related  to 
feedback  seeking  is  also  to  be  expected.  Those  who  value  external  feedback  more  also 
reported  taking  steps  to  get  more  of  it  Somewhat  surprisingly,  we  found  that  Internal 
Feedback  Confidence  (INFBCON)  also  had  a  significant  positive  regression  weight.  We 
suspect  that  in  the  early  stages  of  performing  a  new,  challenging  task,  people  who  are  high 
on  this  characteristic  tend  to  seek  out  others  for  feedback  about  the  expectations  for  and 
requirements  of  the  task.  That  is,  they  want  to  understand  the  “rules  of  the  game.”  This 
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information  is  necessary  for  establishing  the  internal  standards  that  allows  the  person  to 
self-generate  accurate  internal  feedback. 

Tnstnimentalitv  of  BFITS.  Table  6  shows  the  multiple  regression  of  perceived 
instrumentality  of  BFITS  on  flight  experience,  BFITS  performance,  and  the  individual 
differences.  Flight  experience  had  a  statistically  significant  association  with  perceived 
instrumentality,  accounting  for  5.8%  of  the  variance.  The  negative  regression  weight 
indicates  that  those  with  more  previous  flight  experience  perceived  BFITS  to  be  less 
instrumental  (or  useful)  to  them.  Interestingly,  the  next  variable,  BFTTS  performance,  was 
not  related  to  perceived  instrumentality,  after  controlling  for  previous  flight  experience.  In 
Table  2,  it  had  shown  a  statistically  significant,  positive  zero-order  correlation  (r  =  .17). 
Thus,  differences  in  BFITS  performance  do  not  appear  to  have  an  influence  on  the 
perceived  usefulness  of  BFITS.  This  suggests  that  perceived  usefulness  is  not  a  function 
of  how  well  a  person  did  in  BFTTS. 


Insert  Table  6  about  here 


Of  the  individual  difference  variables,  we  see  that  External  Feedback  Propensity 
again  has  a  significant  effect  on  perceived  instrumentality  suggesting  that  people  who  like 
external  feedback  may  have  seen  the  extensive  feedback  from  BFITS  as  valuable.  But 
more  interesting  is  the  interaction  between  BFITS  performance  and  External  Feedback 
Propensity  which  accounts  for  an  additional  5.7%  of  the  variance  in  perceived 
instrumentality.  This  interaction  is  described  by  Figure  1. 


Insert  Figure  1  about  here 
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Of  the  group  that  performed  more  poorly  on  BFITS  (e.g.  more  trials  to 
completion),  it  appears  that  higher  levels  of  External  Feedback  Propensity  leads  to 
perceptions  of  greater  instrumentality.  There  is  no  difference  in  instrumentality  perceptions 
for  good  performers.  Individuals  who  are  high  in  External  Feedback  Propensity  may  not 
be  as  bothered  by  the  negative  feedback  from  the  larger  number  of  trials  and  continue  to  see 
value  in  BFITS  as  contrasted  to  individuals  who  have  a  lower  External  Feedback 
Propensity.  An  additional  plausible  explanation  is  that  the  high  External  Feedback 
Propensity  individuals  seek  feedback  beyond  that  provided  by  the  BFITS  program.  They 
appear  to  consult  with  instructors  or  classmates  and  this  additional  feedback  helps  them 
keep  their  BFITS  performance  in  perspective.  This  explanation  is  supported  by  the  earlier 
described  relationship  between  External  Feedback  Propensity  and  feedback  seeking. 

This  finding  is  important  because  it  points  out  that  the  individual  difference  variable 
seems  to  have  its  effect  for  poorly  performing  individuals.  From  a  motivational 
perspective,  it  is  important  that  these  individuals  continue  to  see  value  in  using  the 
simulator.  Otherwise  it  becomes  an  exercise  to  be  completed  with  little  reason  to  focus  on 

actually  learning  from  it.  Therefore,  it  is  these  poorly  performing  individuals  for  whom  we 

/ 

tend  to  target  training  interventions.  Because  External  Feedback  Propensity  plays  a  role  in 
the  instrumentality  belief  for  poor  performers,  this  further  suggests  that  the  training 
organization  must  continue  to  provide  feedback  opportunities  from  other  people  such  that 
high  external  propensity  individuals  can  obtain  such  feedback.  On  the  other  hand,  it  also 
suggests  that  the  training  organization  must  be  more  proactive  in  dealing  with  the  low 
External  Propensity  individuals.  They  are  less  likely  to  voluntarily  choose  to  avail 
themselves  of  existing  external  feedback  opportunities.  Nor  is  it  as  certain  that  they  will 
see  the  value  in  requiring  them  to  engage  in  such  feedback  sessions.  More  will  be  said 
about  this  after  further  results  are  presented. 
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Stress  Experienced  During  BFITS.  BFITS  performance  was  the  only  main  effect 
that  approached  significance  (p  <.09).  There  were  no  main  effects  for  the  individual 
differences.  However,  reported  stress  was  related  to  an  interaction  between  Internal 
Feedback  Propensity  and  BFITS  performance  which  accounted  for  7.4  percent  of  the 
variance.  These  results  are  shown  in  Table  7  and  the  interaction  in  Figure  2. 


Insert  Table  7  and  Figure  2  about  here 


The  finding  that  stress  is  the  outcome  of  an  interaction  between  a  situation,  in  this 
case  BFITS  performance,  and  an  individual  difference  is  consistent  with  the  literature  on 
stress  that  has  noted  that  it  is  how  individuals  react  to  situations  that  produce  (or  result  in) 
the  experience  of  stress.  So  while  doing  poorly  on  BFITS  was  somewhat  related  to  stress, 
the  real  finding  is  in  the  interaction. 

In  this  case,  those  students  who  tended  to  reconcile  feedback  differences  in  favor  of 
their  own  internal  feedback  (i.e.,  high  on  Internal  Feedback  Propensity)  reported  a 
moderate  level  of  stress  regardless  of  their  BFITS  performance  level.  In  contrast,  the 
stress  reported  by  those  low  on  this  individual  difference  was  directly  related  to  BFITS 
performance.  For  those  who  did  well  on  BFITS,  their  stress  level  was  the  lowest  repotted, 
while  those  who  did  poorly  repotted  the  highest  level  of  stress.  It  would  appear  that  those 
m  this  latter  group  tended  to  see  poor  BFITS  performance  as  significantly  more  threatening 
than  those  who  looked  more  internally  for  how  they  were  doing  when  there  were 
differences  between  their  own  and  BFl’I'S-based  feedback.  Performing  poorly  on  BFITS 
would  have  produced  a  great  deal  of  negative,  outcome  feedback.  Those  high  in  INFBPR 
seemed  to  be  buffered,  at  least  somewhat,  from  finding  this  to  be  a  very  stressful  situation. 
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Flight  Performance  Results 


Up  to  this  point,  we  have  focused  on  results  related  to  the  BFl'l'S  training 
simulator.  Participant  performance  and  reactions  to  the  simulator  appear  to  be  related  to 
several  of  the  feedback-related  propensities.  Next,  we  will  describe  the  results  for  the  in¬ 
flight  training,  specifically  the  number  of  hours  it  took  to  satisfy  the  requirements  for  the 
private  pilot’s  license. 

Time  To  Private  Pilot's  License.  Time  to  private  (TMPRVT)  was  significantly 
related  to  BFITS  performance  and  an  interaction  between  BFITS  performance  and  External 
Feedback  Propensity  (EXFBPR).  The  BFITS  main  effect  accounted  for  24.3  percent  of 
the  variance  and  the  interaction  contributed  another  5.1  percent.  The  results  are  shown  on 
Table  8  and  the  interaction  is  depicted  on  Figure  3. 


Insert  Table  8  and  Figure  3  about  here 


It  should  be  noted  that  the  results  for  time  to  private  are  based  solely  on  the  students 
who  had  little  or  no  flight  experience,  thus  reducing  the  sample  size  to  86  and  this  is  why  ^ 
flight  experience  was  not  included  in  this  analysis.  However,  even  on  this  reduced 
sample,  the  results  for  BFITS  are  very  encouraging.  Performance  on  BFITS  was  directly 
related  to  how  long  it  took  the  students  to  earn  their  private  pilot  license.  Therefore, 
finding  ways  to  help  students  perform  better  on  the  simulator  can  translate  into  better 
cockpit  performance. 

The  interaction  between  BFITS  and  External  Feedback  Propensity  points  to  a 
problem  for  those  who  performed  poorly  on  BFITS  and  were  less  externally  oriented.  It 
appears  that  somehow  those  high  on  EXFBPR  were  better  able  to  reduce  the  negative 
impact  of  "failure"  on  BFITS.  Results  reported  above  provide  a  hint  as  to  why  these 
findings  occurred.  It  was  previously  shown  that  feedback  seeking  was  positively  related  to 
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both  BFITS  performance  and  EXFBPR.  As  such,  it  appears  that  those  high  on  EXFBPR 
and  who  were  having  trouble  were  more  likely  to  get  additional  feedback  from  others.  K 
we  assume  that  some  of  the  feedback  they  received  would  have  provided  comparative 
feedback,  these  students  would  have  seen  that  they  were  not  the  only  ones  running  into 
problems  and  could  have  derived  support  from  such  comparisons.  Also,  it  is  likely  that 
those  from  whom  they  sought  feedback  may  have  provided  direct  encouragement  as  well. 
This  "outside"  information  may  have  helped  the  students  to  put  their  difficulties  into  a  more 
positive  and  useful  perspective. 

The  effects  of  early  simulator  performance.  Finally,  we  investigated  whether  or  not 
the  above  findings  concerning  the  interaction  of  individual  differences  and  BFITS 
performance  could  be  identified  earlier  on  in  training.  If  the  trends  of  certain  people  who 
perform  poorly  on  BFITS  and  will  ultimately  perform  poorly  in  the  cockpit  (relative  to 
others)  could  be  identified  earlier,  it  would  suggest  that  early  identification  and  possible 
interventions  may  allow  us  to  better  tailor  training  to  these  trainees.  Since  the  first  4 
lessons  of  the  simulator  concerned  basic  flight  maneuvers  (e.g.,  climbs  and  turns),  we 
computed  a  “Phase  I  BFITS  performance”,  BFITS  1,  measure  consisting  of  the  total  trials 
to  criterion  for  these  early  lessons  and  repeated  the  above  analysis. 

We  found  that  trials  to  criterion  in  the  basic  procedure  phase  of  BFITS  was 
positively  related  to  hours  to  private  pilot  license  (r  =  .33;  p  <  .01  compared  to  r  =.51  for 
total  BFITS  performance).  Thus,  those  students  who  performed  well  early  in  simulator 
training  tended  to  perform  well  during  in-flight  training. 

As  in  the  previous  analyses,  we  conducted  a  hierarchical  multiple  regression  of 
TMPRVT  on  BFITS  1  and  the  individual  difference  variables.  Table  9  reports  the  results  of 
this  analysis.  In  step  1,  BFITS  1  performance  was  a  significant  predictor  of  in-flight 
performance.  BFITS  1  performance  explained  17.3%  of  the  variance  in  time  to  private.  In 
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the  next  step  of  the  analysis,  the  six  individual  differences  did  not  explain  a  significant 
increment  in  time  to  private. 


Insert  Table  9  about  here 


Finally,  in  step  three  of  the  analysis,  the  interaction  of  Propensity  to  Self  Reinforce 
and  BFITS  performance  entered  as  a  significant  predictor  of  in-flight  performance  (p  <  .05) 
and  explained  an  additional  4.6  %  of  the  variance.  Figure  4  shows  a  graph  of  this 
interaction.  For  persons  who  have  a  high  Propensity  to  Self  Reinforce  their  performance 
experiences,  there  is  no  relationship  between  BFITS  performance  and  flight  performance. 
However,  for  those  low  in  this  individual  difference,  there  is  a  strong  relationship  between 
BFITS  performance  and  flight  performance.  Thus,  those  with  a  low  Propensity  to  Self- 
Reinforce  who  perform  poorly  on  the  simulator  also  performed  poorly  during  in-flight 
training.  However,  when  those  with  a  low  Propensity  to  Self-Reinforce  performed  well  on 
the  simulator,  they  also  did  well  during  in-flight  training. 


Insert  Figme  4  about  here 


Probably  the  most  important  finding  in  the  analysis  of  early  BFITS  performance 
and  the  individual  differences  was  the  sizable  interaction  between  Propensity  to  Self- 
Reinforce  and  performance  on  time  to  private.  In  the  introduction  to  this  phase  we 
suggested  that  excessive  trials  in  the  simulator  are  likely  to  be  interpreted  as  negative 
feedback  by  the  trainee.  With  no  instructor  present  to  offer  encouraging  words,  the  poor 
performing  trainee  may  experience  dysfunctional  stress,  negative  thoughts,  frustration,  and 
other  distracting  emotions  that  if  no  counteracted,  significantly  reduce  learning  during  these 
trials.  For  the  person  who  has  a  propensity  to  self-reinforce,  they  are  providing  a 
counteracting  force  to  the  negative  feedback  from  the  simulator,  which  may  enable  them  to 
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maintain  higher  levels  of  concentration  and  confidence  which  facilitate  learning.  Much  as 
the  External  Feedback  Propensity  may  be  important  for  obtaining  the  support  and 
perspective  from  other  people  during  BFITS,  the  Propensity  to  Self-Reinforce  helps  the 
trainee  remained  focused  during  early  lessons. 
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Discussion  of  field  study  results 


In  order  to  organize  the  many  results  derived  from  the  field  study  phase  of  our 
research,  a  summary  has  been  prepared  and  is  shown  in  Table  10.  Overall,  it  can  be  seen 
that  of  the  six  feedback-related  individual  differences  that  were  developed  and  tested,  four 
of  them  were  significant  predictors  of  the  outcomes  investigated  with  the  COMAIR 
Aviation  Academy  flight  students.  These  result  occurred  after  flight  experience  and  BFITS 
performance  were  already  taken  into  account  Internal  Feedback  Confidence,  Internal 
Feedback  Propensity ,  and  External  Feedback  Propensity  were  significant  main  effects, 
with  the  latter  two  propensities  also  interacting  with  BFITS  performance.  Self- 
Reinforcement  was  the  one  individual  difference  that  was  only  significant  as  part  of  an 
interaction. 


Insert  Table  10  about  here 


The  number  of  significant  relationships  found  for  the  individual  differences  is 
extremely  encouraging  and  points  to  the  important  role  of  how  individuals  respond  to 
performance  feedback,  especially  in  a  training  setting.  It  is  also  worth  noting  that  with  six 
outcomes  investigated  (Time  to  Private  was  tested  twice  -  in  relation  to  total  BFITS 
performance  and  then  with  performance  in  the  first  phase  of  BFITS),  three  of  the  individual 
differences  were  involved  in  four  interactions.  This  provides  support  for  the  expectation 
that  the  individual  differences  will  often  not  show  up  as  simple  main  effects,  but  that  their 
real  power  is  revealed  in  relation  to  things  like  current  and  past  performance. 

Within  this  setting.  External  Feedback  Propensity  was  the  most  consistent 
predictor  of  reactions  to  BFITS.  It  was  positively  related  to  the  desire  for  external 
feedback,  feedback  seeking,  and  perceived  instrumentality  of  BFITS.  These  findings  are 
consistent  with  other  research  we  have  conducted  wherein  those  who  reported  wanting 
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more  feedback  have  taken  steps  to  acquire  it  Therefore,  it  is  not  surprising  that  in  this 
setting,  where  there  was  no  formal  external  feedback,  those  high  on  External  Feedback 
Propensity  reported  desiring  external  feedback  and  acquiring  external  feedback  on  their 
own. 

Furthermore,  External  Feedback  Propensity  had  a  significant  main  effect  and 
interacted  with  BFTTS  performance  to  predict  instrumentality  perceptions.  This  is  the  first 
time  that  we  have  investigated  instrumentality  in  relation  to  the  feedback-related  individual 
differences.  These  results  point  not  only  to  a  greater  acceptance  of  training  by  those 
externally  oriented,  but  also  that  this  may  motivate  them  to  exert  greater  effort  in  learning 
from  BFITS.  Moreover,  External  Feedback  Propensity  interacted  with  Bbll  S 
performance  to  predict  time  to  private  pilot’s  license.  In  this  case,  those  high  in  this 
propensity  and  who  had  difficulty  in  BFITS  were  better  able  to  use  the  BFITS  experience 
in  improving  their  cockpit  performance. 

Internal  Feedback  Confidence  was  negatively  related  to  BFITS  performance  (i.e., 
the  higher  the  confidence,  the  better  the  performance)  and  feedback  seeking  (the  higher  the 
confidence,  the  more  feedback  seeking).  The  first  finding  was  anticipated,  but  the  second 
one  was  not.  Normally,  those  who  are  self-confident  in  their  feedback  generating  abilities 
would  not  be  expected  to  do  feedback  seeking.  However,  given  the  training  setting,  we 
understand  this  finding  to  represent  a  need  for  information  with  which  to  build  appropriate 
evaluation  standards.  Therefore,  those  who  are  high  on  Internal  Feedback  Confidence  will 
take  steps  to  be  able  to  self-evaluate  when  necessary. 

Internal  Feedback  Propensity  was  positively  related  to  BFITS  performance  (i.e., 
the  higher  the  propensity,  the  poorer  the  performance).  As  noted  earlier,  this  has  been 
found  in  a  previous  training  setting.  Those  who  prefer  their  own  internal  feedback, 
especially  when  it  differs  from  external  feedback,  may  block  out  valuable  information.  In 
addition.  Internal  Feedback  Propensity  interacted  with  BFTTS  performance  to  predict 
stress.  In  this  case,  an  internal  feedback  orientation  helped  reduce  the  experience  of  stress 
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when  faced  with  performance  difficulties.  As  such,  Internal  Feedback  Propensity  appears 
to  be  a  "two  edged  sword"  that  can  help  reduce  stress,  but  also  reduce  learning  during 
training. 

Self-Reinforcement  was  not  a  significant  main  effect  for  any  of  the  outcomes 
tested,  but  did  interact  with  the  first  phase  of  BFITS  performance  to  predict  time  to  the 
private  pilot's  license.  In  this  case,  those  who  are  high  on  this  individual  difference  were 
better  able  to  learn  from  BFITS  even  when  encountering  performance  difficulties. 

As  previously  noted,  two  of  the  feedback-related  individual  differences.  Internal 
Feedback  Ability  and  External  Feedback  Propensity  -Negative  did  not  play  significant 
roles  in  this  study.  However,  this  does  not  suggest  that  they  should  be  dropped  from 
further  consideration.  We  recognize  that  this  study  was  performed  in  a  very  particular 
training  situation.  While  simulators  are  becoming  more  and  more  common,  it  is  still  not  the 
most  prevalent  way  to  train  people.  A  simulator  is  only  one  type  of  source  for  performance 
information.  There  were  no  external  sources  providing  negative  feedback  as  part  of  the 
training  and  the  sources  sought  out  by  the  student  pilots  most  likely  provided  either 
process,  comparative,  or  positive  feedback.  In  such  a  setting  it  is  not  surprising  that 
External  Feedback  Propensity  -Negative  was  not  related  to  reactions  or  performance. 

Apparendy,  one’s  Internal  Feedback  Ability  also  was  not  relevant  in  this  situation. 
However,  if  the  training  setting  was  significantly  different,  or  a  non-training  setting  was 
used,  the  set  of  feedback-related  individual  differences  that  function  as  predictors  could 
change  significantly.  The  fact  that  four  of  the  six  were  shown  in  this  study  to  add 
significantly  to  our  explanatory  power  of  important  outcomes  is  noteworthy  and  an 
excellent  based  point  from  which  to  further  our  knowledge  of  these  feedback-related 
individual  differences. 
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SUMMARY  AND  CONCLUSIONS 

This  program  of  research  was  proposed  in  order  to  explore  the  intersection  of  two 
important  concepts  thought  to  affect  human  performance,  especially  in  learning,  training,  or 
skill-acquisition  settings,  namely,  performance  feedback  and  individual  differences  in 
generating,  processing  or  reacting  to  such  feedback.  While  there  exists  an  extensive 
literature  on  the  nature  and  beneficial  effects  of  performance  feedback,  little  is  known  about 
how  particular  individuals  react  to,  or  shape  their  feedback  environment.  The  thrust  of  our 
program  has  been  to  identify  and  assess  a  unique  set  of  individual  differences  which 
characterize  how  people  interact  with  their  feedback  environment.  We  call  this  a  domain- 
specific  approach  to  individual  differences  because  global  assessments  have  not  been  as 
potent  as  one  might  have  hoped  for  in  identifying  person-situation  interactions  which  are  so 
important  for  understanding  human  performance. 

By  domain-specific,  we  mean  explicitly  making  a  fidelity-bandwidth  tradeoff,  such 
that  we  accept  lower  cross-situational  predictive  power  in  return  for  better  understanding 
behaviors  in  a  nairower  set  of  situations,  i.e.,  situations  in  which  performance  feedback  is 
thought  to  be  important  for  self-regulation,  performance  improvement,  and/or  motivation. 
Building  on  everyday  experiences  that  people  obviously  differ  in  their  proclivities  toward 
seeking  feedback,  valuing  it,  accepting  it,  or  even  providing  it  to  themselves,  plus  some  of 
our  eaiiier  research  findings  which  suggested  that  such  proclivities  could  be  assessed  in 
terms  of  external  or  internal  orientations  toward  feedback,  we  set  out  to  demonstrate  two 
things:  a)  that  a  set  of  feedback-related  individual  differences  could  be  identified,  and 
sound  measures  of  these  developed;  and  b)  that  such  differences  relate  to  performance- 
related  phenomena  in  a  systematic  and  potentially  important  manner. 

Through  a  series  of  field  and  lab  studies,  as  well  as  extensive  psychometric  work, 
we  identified  six  dimensions  along  which  individuals  can  vary  in  their  feedback 
propensities.  We  showed  that  the  various  distinctions  are  more  complicated  than  just  a 


single,  bi-polar  dimension  of  intemal-exteraal.  We  showed  that  internal  orientations 
toward  performance  feedback  can  be  dimensionalized  according  to  the  tendency  to  value 
internal  feedback,  the  tendency  to  self-reinforce  one’s  self  for  performance,  the  ability  to 
self- assess,  and  the  confidence  one  has  in  one’s  self  assessments.  We  showed  that 
external  orientations  toward  performance  feedback  could  be  distinguished  in  terms  of 
whether  one  values  feedback  one  expects  to  be  largely  neutral  or  positive,  or  whether  one 
values  external  feedback  even  if  it  is  negative. 

We  engaged  in  a  variety  of  construct  validation  attempts  with  both  the  originally- 
proposed  3-dimension  scheme  and  the  subsequently  developed  6-dimension  scheme.  In  aU 
cases  we  showed  that  our  measures  are  theoretically  consistent  with  other  measures  of 
individual  differences,  but  not  redundant  with  them.  We  also  showed,  in  a  variety  of 
settings,  and  using  widely  different  methodologies  and  populations  that  our  measures  are 
associated  with  differential  performance  under  different  feedback  conditions,  as  well  as 
with  attitudinal  and  cognitive  reactions  to  feedback  situations.  For  example,  we  found  that 
external  orientation  to  feedback  explained  people’s  seeking  of  feedback  from  peers  and 
supervisors;  we  found  that  internal  propensities  were  related  to  task  mastery  in  a  training 
session  where  not  much  feedback  was  available  to  the  trainee;  we  found  differences  in  the 
asking  for  process  and  outcome  feedback  when  learning  a  novel  task;  we  found  differences 
in  trainees  assessments  of  their  instructor  based  on  the  trainees’  feedback  orientations. 

While  the  above  could  be  construed  as  things  we  learned  through  a  series  of 
exploratoiy  studies  aimed  at  pinning  down  some  of  the  measures  and  demonstrating  their 
utility  in  various  settings,  our  final  field  demonstration  was  intended  to  be  a  much  more 
comprehensive  test  of  the  potential  appUcations  of  this  work.  The  field  demonstration  may 
be  said  to  be  the  final  phase  of  exploratory  research.  We  have  developed  sufficient 
understanding  of  the  behavior  and  application  of  our  measures  to  allow  us,  and  others  to 
move  to  hypothesis-testing  research.  This  field  demonstration  also  bridged  our  more  basic 
research  with  important  applications  which  might  make  use  of  our  work. 
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Several  impoitant  points  should  be  noted  about  this  final  phase  of  our  work.  First, 
we  were  able  to  demonstrate  important  differences  in  people’s  response  to  a  real-world 
training  situation  in  which  the  nature  and  amount  of  feedback  was  directly  based  on  the 
design  of  the  training  protocol  and  the  performance  of  the  trainee.  Second,  we  found 
differences  between  types  of  people  for  a  variety  of  attitudinal  measures,  self-descriptions 
of  behavior,  as  well  as  actual  performance.  Third,  all  responses,  attitudinal,  self- 
descriptive  behaviors  and  performance  were  temporally  removed  from  the  assessment  of 
the  individual  differences,  thus  systematic  relationships  are  not  likely  to  be  a 
methodological  artifact.  Fourth,  our  performance  measures  were  hard  measures, 
independently  obtained,  and  of  great  importance  to  the  individuals  and  the  organization. 
Specifically,  differences  in  many  hours  of  training  required  to  achieve  a  given  level  of 
proficiency,  when  such  training  costs  $1 20/hour,  not  to  mention  motivational  and 
scheduling  problems  which  occur  when  performance  is  sub-par,  are  clearly  significant. 
Fifth,  our  measures  of  individual  differences  were  shown  to  be  potent  predictors  of 
behavior,  even  when  competed  against  traditionally  strong  predictors  such  as  similar  past 
performance. 

Finally,  in  a  tme  demonstration  of  interactionist  perspectives  on  human  behavior, 
person-situation  interactions  were  repeatedly  found  in  our  research.  This  last  point 
strongly  suppoits  the  move  toward  domain-specific  measures  of  individual  differences.  If 
one  specifies  the  domain  and  theoretically  develops  conceptualizations  of  what  kinds  of 
individual  differences  are  likely  to  be  evoked  by  such  domains,  one  is  likely  to  make  more 
progress  than  either  assuming  that  situations  are  all  powerful  and  variances  in  behavior  are 
random,  or,  investigating  individual  differences  which  do  not  theoretically  map  onto  the 
situations  being  studied.  Statistical  interactions  of  the  type  we  have  found,  in  terms  of  both 
their  consistency  and  variance  explained  are  relatively  unique  in  social  science.  Thus,  we 
hope  that  we  have  made  a  contribution  not  just  to  the  understanding  of  performance 
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feedback  phenomena,  but  also  to  demonstrating  alternative  ways  of  pursuing  individual 
differences  research. 

In  terms  of  implications  for  the  future,  we  believe  that  we  have  identified  many 
potentially  important  areas  which  may  be  investigated  based  on  our  foundation.  Briefly, 
we  will  illustrate  some  of  these  implications.  Our  research  has  implications  for  the  design 
of  training  programs  so  that  feedback  provided  or  made  available  may  be  better  matched  to 
the  needs  of  the  individual  trainee.  This  may  be  especially  important  in  feedback- 
constrained  situations,  such  as  simulators  or  self-paced  instruction.  In  these  situations  we 
may  eventually  be  able  to  augment  the  currently  available  feedback  through  redesign  of  the 
technology  or  the  addition  of  alternative  sources  of  feedback  such  as  instructors.  We  may 
also  be  able  to  create  feedback  “menus,”  such  that  individuals  may  seek  feedback  (e.g., 
outcome  only,  or  process)  which  better  suits  their  learning  habits. 

While  we  do  not  want,  at  this  point,  to  promote  our  work  as  having  organizational 
selection  implications,  it  may  not  be  too  early  to  investigate  job  assignment  or  placement 
implications.  Certain  jobs  or  assignments,  by  their  nature,  have  more  or  less  inherent 
feedback,  of  varying  quality  or  ambiguity,  available  to  performers.  Conceivably,  our 
predispositions  may  be  useful  in  predicting  who  will  prosper  or  flounder  in  these  different 
types  of  situations.  Another  application  of  our  research,  supported  by  our  examination  of 
the  early  simulator  perfonnance  of  trainees,  may  be  in  monitoring  the  performance  of 
trainees  for  early  signs  of  difficulties.  Our  research  suggests  that  some  individuals  will  be 
more  handicapped  by  early  poor  performance  during  training  than  others. 

Finally,  our  research  opens  the  possibility  that  self-assessment  and  self-regulation, 
so  important  for  the  mastery  and  maintenance  of  many  behaviors  in  organizations,  may  be, 
to  some  degree,  trainable.  If  individuals  could  be  trained  to  self-assess  on  a  particular  task, 
have  their  confidence  built  that  such  assessments  are  veridical,  and  develop  the  ability  to 
self-reinforce  both  the  act  of  self-assessment  and  their  performance,  we  may  be  able  to 
achieve  levels  of  self-regulated,  self-improving,  and  self-maintaining  performance  we  so 


34 


want  in  many  situations.  Unfortunately,  many  of  our  current  training  procedures,  trying  as 
they  are  to  minimize  time  and  resources  to  achieve  proficiency,  may  be  substituting  short- 
tenn  criterion  attainment  for  long  term  performance  maintenance. 

In  summary,  we  believe  that  we  have  accomplished  everything  we  set  out  to 
accomplish.  In  the  true  spirit  of  programmatic  research,  we  have  cumulatively  built  upon 
our  various  research  efforts  to  arrive  at  some  interesting,  and  potentially  important 
conclusions  about  person  variables  associated  with  performance  feedback.  Our  work  has 
been  well  accepted  by  the  scientific  community  in  terms  of  journal  publications,  book 
chapters,  and  scientific  presentations,  and  we  foresee  several  important  contributions  stiU 
coming  from  this  research.  This  research  has  also  engaged  a  variety  of  doctoral  students, 
with  at  least  three  dissertations  having  been  influenced  by  this  work.  Hopefully  this  will 
lead  to  a  propagation  of  effoits  which  will  speed  up  the  accumulation  of  knowledge  in  this 
area. 
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Table  1 

Descriptive  Statistics  For  All  Study  Variables 
and  Sample  Demographics 


Variable 

Mean 

Standard  E 

AGE 

26.3 

5.66 

SEX 

.07 

.26 

FLTEXP 

2.79 

1.76 

INFBAB 

3.98 

.47 

INFBCON 

3.56 

.61 

SELFRE 

3.78 

.57 

INFBPR 

3.72 

.62 

EXFBPR 

3.88 

.54 

EXPBNE 

3.99 

.58 

BFITSl 

74.22 

57.83 

BFITS 

350.41 

206.68 

Legend 

AGE  -  Student’s  Age  Upon  Entering  COMAIR 
SEX  -  Student's  Sex  (Coded  0  =  Male,  1  =  Female) 

FLTEXP-Flight  Experience 
INFBAB  -  Internal  Feedback  Ability 
INFBCON  -  Internal  Feedback  Confidence 
SELFRE  -  Propensity  to  Self-Reinforce 
INFBPR  -  Internal  Feedback  Propensity 
EXFBPR  -  External  Feedback  Propensity 
EXFBNE  -  External  Propensity  for  Neg.  Feedback 

BFITSl  -  Total  Number  of  Trials  to  Complete  the  First  of  Four  Flight  Segments 
BFITS  -  Total  Number  of  Trials  to  Complete  All  BFITS  lessons 
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Table  2 

Intercorrelations  For  Variables 
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Table  3 

Regression  Results  for  Predicting  BFITS  Performance 


Predictors  Beta 


Step  1 

FLTEXP 

-.41 

Step  2 

INFBAB 

ns 

INFBCON 

-.16 

SELFRE 

ns 

INFBPR 

.17 

EXFBPR 

ns 

EXPBNE 

ns 

.220 


BFITS  =  total  number  of  trials  in  BFITS  simulator 

TMPRIV  =  number  of  hours  to  complete  private  pilot  license 

INFBAB  =  Internal  feedback  ability 

INFBCON  =  Internal  feedback  confidence 

INFBPR  =  Internal  feedback  propensity 

SELFRE  =  Propensity  to  self-reinforce 

EXFBPR  =  External  feedback  propensity 

EXFBNE  =  External  propensity  for  negative  feedback 


r2  Change 


.170 


.050 
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Table  4 

Regression  Results  for  Predicting  Feedback  Desire 


Predictors  Beta 

Step  1 


FLTEXP 

-.37 

Step  2 

BFITS 

.28 

Step  3 

INFBAB 

ns 

INFBCON 

ns 

SELFRE 

ns 

INFBPR 

ns 

EXFBPR 

.17 

EXFBNE 

ns 

r2 

Change 

.140 

.140 

.198 

.057 

.270 


.072 


Legend 

BFITS  =  total  number  of  trials  in  BFITS  simulator 

TMPRJV  =  number  of  hours  to  complete  private  pilot  license 

INFBAB  =  Internal  feedback  ability 

INFBCON  =  Internal  feedback  confidence 

INFBPR  =  Internal  feedback  propensity 

SELFRE  =  Propensity  to  self-reinforce 

EXFBPR  =  External  feedback  propensity 

EXFBNE  =  External  propensity  for  negative  feedback 
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Table  5 

Regression  Results  for  Predicting  Feedback  Seeking 


Predictors 

Step  1 

FLTEXP 

Step  2 
BFITS 

Step  3 

INFBAB 

INFBCON 

SELFRE 

INFBPR 

EXFBPR 

EXFBNE 

Step  4 

Interactions 

Step  5 

FBDESIRE 


Beta 


r2 


-.27 


.074 


ns 

.079 


ns 

.23 

ns 

ns 

.22 

ns 

.162 


ns 


.16  (.07) 


.182 


r2  Change 


.074 


.005 


.083 


.019 


Legend 

BFITS  =  total  number  of  trials  in  BFITS  simulator 

TMPRIV  =  number  of  hours  to  complete  private  pilot  license 

INFBAB  =  Internal  feedback  ability 

INFBCON  =  Internal  feedback  confidence 

INFBPR  =  Internal  feedback  propensity 

SELFRE  =  Propensity  to  self-reinforce 

EXFBPR  =  External  feedback  propensity 

EXFBNE  =  External  propensity  for  negative  feedback 

FBDESIRE=  Desire  for  Feedback  During  BFITS 
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Table  6 

Regression  Results  for  Predicting  Instrumentality 

Ersdictprg  Beta  r2 

Step  1 


FLTEXP 

-.24 

Step  2 

BFITS 

ns 

Step  3 

INFBAB 

ns 

INFBCON 

ns 

SELFRE 

ns 

INFBPR 

ns 

EXFBPR 

.20 

EXFBNE 

ns 

.108 

Step  4 

EXFBPRxBFITS  1.83 

.165 


Legend 

BFITS  =  total  number  of  trials  in  BFITS  simulator 
FLTEXP  =  Flight  Experience 
INFBAB  =  Internal  feedback  ability 
INFBCON  =  Internal  feedback  confidence 
INFBPR  =  Internal  feedback  propensity 
SELFRE  =  Propensity  to  self-reinforce 
EXFBPR  =  External  feedback  propensity 
EXFBNE  =  External  propensity  for  negative  feedback 


r2  Change 


.058 


.006 


.045 


.057 
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Table  7 

Regression  Results  for  Predicting  Stress 
Experienced  During  BFITS 


Predictors  Beta  r2 

Step  1 

FLTEXP  ns 

.00 


Step  2 

BFITS  ns 

.019 

Step  3 


INFBAB 

ns 

INFBCON 

ns 

SELFRE 

ns 

INFBPR 

ns 

EXFBPR 

ns 

EXFBNE 

ns 

Step  4 

BFITS  X 

INFBPR 

^2.25 

Lc.send 

BFITS  =  total  number  of  trials  in  BFITS  simulator 

FLTEXP  =  Flight  Experience 

INFBAB  =  Internal  feedback  ability 

INFBCON  =  Internal  feedback  confidence 

INFBPR  =  Internal  feedback  propensity 

SELFRE  =  Propensity  to  self-reinforce 

EXFBPR  =  External  feedback  propensity 

EXFBNE  =  External  propensity  for  negative  feedback 


r2  Change 


.00 


.019 


.014 


.074 
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Table  8 

Regression  Results  for  Predicting 
Time  to  Private  Pilot's  License 


Predictors 

Beta 

Step  1 

BFITS 

.49 

Step  2 

INFBAB 

ns 

INFBCON 

ns 

SELFRE 

ns 

INFBPR 

ns 

EXFBPR 

ns 

EXFBNE 

ns 

Step  3 
BFITS  X 

EXFBPR  -1.73 


r2 


.243 


.272 


.324 


Legend 

BFITS  =  total  number  of  trials  in  BFITS  simulator 

TMPRIV  =  number  of  hours  to  complete  private  pilot  license 

INFBAB  =  Internal  feedback  ability 

INFBCON  =  Internal  feedback  confidence 

INFBPR  =  Internal  feedback  propensity 

SELFRE  =  Propensity  to  self-reinforce 

EXFBPR  =  External  feedback  propensity 

EXFBNE  =  External  propensity  for  negative  feedback 


r2  Change 


.243 


.030 


.051 
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Table  9 

Regression  Results  for  Predicting  Time  to  Private  from  BFITS  1 


Predictors 

Beta 

r2 

r2  Change 

Step  1 

BFITS  1 

.42 

.173 

.173 

Step  3 


INFBAB 

ns 

INFBCON 

ns 

SELFRE 

ns 

INFBPR 

ns 

EXFBPR 

ns 

EXFBNE 

ns 

.242  .068 

Step  4 

SELFRE  X  BFITS  -2.32 

.288  .046 


Legend 

BFITS  =  total  number  of  trials  in  BFITS  simulator 

TMPRIV  =  number  of  hours  to  complete  private  pilot  license 

INFBAB  =  Internal  feedback  ability 

INFBCON  =  Internal  feedback  confidence 

INFBPR  =  Internal  feedback  propensity 

SELFRE  =  Propensity  to  self-reinforce 

EXFBPR  =  External  feedback  propensity 

EXFBNE  =  External  propensity  for  negative  feedback 
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Feedback  Feedback  Instrument- 

BFITS  Desire  Seeking  ality  Stress  Time  to  Private 
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The  Interaction  of  BFITS  Performance  and  External 
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BFITS  Trials 


The  Interaction  of  BFITS  Performance  and  internal 
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BFITS  Trials 


The  Interaction  of  BFITS  Performance  and  External 
Feedback  Propensity  on  Time  to  Private  Pilot's  License 
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BFITS  Trials 


Interaction  of  BFITS1  Performance  and  Self- 
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BFITS1  Trials 


Appendix  1 

Scales  and  Internal  Consistencies 

Internal  Feedback  Ability  (alpha  =  .85) 

1 .  I  find  that  I  am  usually  a  pretty  good  judge  of  my  own 
performance. 

2.  When  learning  a  task,  I'm  pretty  good  at  identifying  what  parts  I 
have  mastered. 

3.  I  usually  have  a  clear  idea  of  what  I  am  trying  to  do  and  how  well 
I  am  proceeding  towards  my  goal. 

4.  When  performing  a  task,  I  can  usually  tell  early  on  if  it  is  going  to 
turn  out  okay. 

5.  When  I  finish  something,  I  can  usually  tell  right  away  whether  I 
did  it  well  or  not. 

6.  I  know  when  my  work  is  not  up  to  standards  before  anybody  tells 
me. 

7.  When  I  finish  something,  I  can  usually  tell  right  away  if  I  did  not 
do  it  well. 

8.  If  I  make  a  mistake  while  working,  I  can  usually  sense  it 
immediately. 

9.  I  usually  know  why  my  performance  is  slipping. 

10.  When  I  have  done  something  well,  I  know  it  without  other 
people  telling  me  so. 
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Internal  Feedback  Confidence  (alpha  =  .64) 


1.  I'm  often  at  a  loss  as  to  how  to  improve  my  performance.  (R) 

2.  When  I’m  disappointed  in  my  performance,  often  I  don’t  know 
what  I  did  wrong.  (R) 

3.  I  find  that  I  am  not  very  good  at  assessing  my  own  performance 
and  need  to  rely  on  the  inputs  of  others.  (R) 

4.  Until  I  hear  I've  done  a  good  job  from  someone  else,  I  often  don't 
trust  my  own  impressions.  (R) 

5.  When  others'  opinions  about  my  work  are  different  than  my  own, 

I  tend  to  question  my  own  judgment.  (R) 

Internal  Feedback  Propensity  (alpha  =  .73) 

1.  If  you  think  you  have  done  something  well,  don't  let  other  people's 
opinions  to  the  contrary  get  you  down. 

2.  As  long  as  I  think  I  have  done  something  well,  I  am  not  too 
concerned  about  how  other  people  think  I  have  done. 

3.  How  other  people  view  my  work  is  not  as  important  as  how  I 
view  my  own  work. 

4.  People  ought  to  be  more  concerned  with  their  self-image  than  with 
what  other  people  think. 

5.  What  I  think  of  myself  and  my  work  is  more  important  to  me  than 
what  others  think. 
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Propensity  to  Self-Reinforce  (alpha  =  .76) 

1.  I  tend  to  celebrate  my  work  accomplishments. 

2.  I  tend  to  give  myself  a  pat  on  the  back  for  a  job  well  done. 

3.  After  finishing  a  project  that  turned  out  well,  I  tend  to  reflect  on 
what  went  right. 

4.  I  like  to  step  back  and  reflect  on  a  job  well  done. 

5.  After  completing  a  project,  I  like  to  spend  some  extra  time 
reflecting  on  those  things  that  I  did  particularly  well. 


External  FeedbackPropensity  (alpha  =  .71) 

1.  It  is  very  important  for  me  to  know  what  people  think  of  my 
work. 

2.  Even  though  I  may  think  I  have  done  a  good  job,  I  feel  a  lot  more 
confident  of  it  after  someone  else  tells  me  so. 

3.  Even  when  I  think  that  I  could  have  done  something  better,  I  feel 
good  when  other  people  think  well  of  what  I  have  done. 

4.  I  like  getting  frequent  feedback  from  others  regarding  my 
performance. 

5.  I  like  being  told  how  well  I  am  doing  on  a  project. 

6.1  don't  like  going  for  long  periods  of  time  without  getting  feedback 
concerning  my  performance. 


57 


External  Propensity  -  Negative  Feedback  (alpha  =  .77) 

1.  I  like  receiving  feedback  that  identifies  where  my  performance 
has  been  deficient. 

2.  I  seek  out  reactions  to  my  work  even  if  I  think  they  might  be 
negative. 

3.  I  look  for  opportunities  to  get  comments  on  my  work,  even  if  those 
won't  be  favorable. 

4.  I  seek  other's  assistance  in  figuring  out  how  to  improve  my 
performance. 

Feedback  Seeking  (alpha  =  .84) 

1.  I  talked  with  other  BFITS  trainees  about  any  difficulties  I 
encountered. 

2.  I  asked  for  help  from  other  BFITS  trainees. 

3.  I  compared  my  progress  through  BFITS  lessons  with  that  of  other 
BFITS  trainees. 

4.  I  asked  others  how  far  along  they  were  with  BFITS. 

5.  I  shared  my  experiences  with  other  BFITS  trainees. 

6.  I  compared  my  BFITS  performance  (e.g.  number  of  times  it  took  to 
get  through  the  lessons)  with  others. 

7.  I  talked  with  others  (including  individuals  not  doing  BFITS)  about 
the  usefulness  of  BFITS. 
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Feedback  Desire  (alpha  =  ,74) 


1.  I  wished  I  had  someone  working  with  me  to  provide  additional 
help. 

2.  I  would  have  liked  access  to  an  instructor  to  help  me  through 
specific  maneuvers. 

3.  I  would  have  liked  for  someone  to  periodically  review  my 
progress  on  BFITS. 

Instrumentality  (alpha  =  ,93) 

1.  I  believe  that  my  BFITS  experience  will  be  helpful  in  my  flight 
training. 

2.  BFITS  helped  me  identify  potential  weaknesses  in  my  flight  skills. 

3.  BFITS  will  save  me  flight  lesson  time. 

4.  BFITS  will  make  me  a  better  pilot. 

Stress  (  alpha  =  .60) 

1.  I  found  BFITS  flight  lessons  to  be  frustrating 

2.  After  the  flight  lessons  on  BFITS,  it  would  take  me  some  time  to 
"come  down." 

3.  I  felt  stress  while  going  through  BFITS  flight  lessons. 
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